The Weights Can Be Harmful: Pareto Search versus Weighted Search in Multi-objective Search-based Software Engineering

In presence of multiple objectives to be optimized in Search-Based Software Engineering (SBSE), Pareto search has been commonly adopted. It searches for a good approximation the problem’s Pareto-optimal solutions, from which stakeholders choose most preferred solution according their preferences. However, when clear preferences (e.g., set weights that reflect relative importance between objectives) are available prior search, weighted is believed first choice, since it simplifies via converting original multi-objective problem into single-objective one and enables focus on what only interested in. This article questions such “ ” belief. We show can, fact, harmful process even Specifically, we conduct large-scale empirical study consists 38 systems/projects three representative SBSE problems, together with two types budget nine sets weights, leading 604 cases comparisons. Our key finding reaches certain level quality by consuming relatively less resources at early stage search; however, significantly better than its counterpart majority time (up 77% cases), as long allow sufficient, but not unrealistic budget. beneficial result, discovers potentially new “rule-of-thumb” community: Even available, recommended always consider default provided more important. Weighted contrast, should resource/search limited, especially expensive problems. This, other findings actionable suggestions article, allows us codify pragmatic comprehensive guidance choosing under circumstance available. All code data can accessed https://github.com/ideas-labo/pareto-vs-weight-for-sbse .